HCL Lotus Expeditor wiki: Using the Lotus Expeditor Integrator Platform: Using Lotus Expeditor Integrator Introduction

Using Lotus Expeditor Integrator Introduction

Added by ~Zach Cisnupulader | Edited by ~Emile Xannilynivu on May 31, 2013 | Version 3

Actions ▼

Abstract

Abstract

No abstract provided.

Tags: Integrator

ShowTable of Contents

HideTable of Contents

1 Identification
2 Solution Overview Example
- 2.1 Component Overview
- 2.2 Example Business Use Cases for Retail - Overview
3 References
4 Notation

Identification

This document belongs to a series of documents describing the IBM® Lotus® Expeditor integrator package which is based on the IBM product Lotus Expeditor Client for Desktops. It comprises information about the usage of the Expeditor integrator software components and their integration in the existing business process flows.

The Expeditor integrator Users Guide document captures all information which is required for seamless integration of the Expeditor integrator into an enterprise’s Application Integration back-end (Backbone Enterprise Bus). IT Specialists and Developers responsible for the business process implementation will find appropriate information about the integration options for their existing IT landscape (e.g. message types and formats etc. for the existing Enterprise Service Bus, Monitoring Systems).

Solution Architects focusing on the design of the Business Use Cases will also find tips how the Expeditor integrator can be adjusted to changed requirements and new business use cases.
A second document, the Installing and Configuring the IBM Lotus Expeditor integrator platform, will further explain how the Expeditor integrator can be installed, configured, maintained and operated in test and production environments (see Ref_1).

Solution Overview Example

Figure 1 shows an architectural overview diagram example including the nearby systems of an overall IBM Lotus Expeditor integrator solution for retail enterprises. The solution example is based on existing retail customer requirements and business use cases which drove the development of Expeditor integrator based on the IBM® Lotus® Expeditor Client for Desktops product (see red box in Figure 1).
In this Users Guide, it is mainly focused on the integration options of the Expeditor integrator with existing enterprise back-end systems.

Figure 1 - Lotus Expeditor integrator Architectural Overview example for retail solutions

The Lotus Expeditor integrator solution can extend the reach of an existing Application Integration back-end based on a common Backbone Service Bus (e.g. WebSphere® Message Broker and WBI Adapter for Flat Files) to remote locations with restricted hardware and software resources.
In the given retail example, data exchange between the 4690 point of sales controllers and the Backbone Service Bus is usually done via FTP and file system access today. Expeditor integrator replaces the existing process by providing:

Reliable data exchange between enterprise Backbone Service Bus and remote locations (e.g. stores).
Centralized management (monitoring, configuration management & software distribution).
Near zero operating effort for the integration infrastructure at the remote location (e.g. store infrastructure such as PoS and scale systems).
Local file access support for local and mounted file systems as well as FTP, FTPs, SSH protocol support at the remote location.
Local data access through JDBC and REST data adapters at the remote location.

The Expeditor integrator was developed based on open standards (e.g. OSGi®, Eclipse®).
Figure 1 also shows the core components of IBM Lotus Expeditor integrator. The main processing unit is the Expeditor integrator Access Control Service (ACS) which manages Flows of Activities. These flows can be manually configured to fulfill distinct (business) use cases. The Resource Monitor monitors given resources of different types through the different Resource Adapters (e.g. FTP Adapter). It fires events for triggering ACS Flows for configurable states of a given resource (e.g. when a defined file appears in a defined directory an event is published which triggers an ACS Flow). The IBM Lotus Expeditor micro broker is the message provider of the Expeditor integrator which ensures performing, reliable and secure data exchange with the back-end or other local resources. Monitoring Services create (configurable) standards based CBE events which can be either sent to the back-end messaging system or to other (business) monitoring systems such as IBM Tivoli Enterprise Console and IBM Business Monitor.
Please, refer to chapter 1.2.1 Component Overview for further details about the Expeditor integrator components.

Component Overview

The modular design of the Expeditor integrator is based on the OSGi® and the Eclipse® Standard (www.osgi.org, www.eclipse.org). This was approach was chosen in order to achieve the required stability of a remotely running application, to address limited available resources on the remote server (e.g. a retail store server) and to get remote control options.

Figure 2 shows the components of the Expeditor integrator.

ACS (Expeditor integrator Application Control Service)
The ACS is the core component of the Expeditor integrator. The Expeditor integrator Application Control Service is a (more general) central processing component for all incoming and outgoing requests/messages in Expeditor integrator. It interprets the requests/messages and triggers further actions to orchestrate a use-case within a transactional unit of work. The successful execution of all ACS Activities within the use-case flow leads to the committing of the containing unit of work within the Expeditor integrator’s Transaction Service or a rollback otherwise.
The ACS Service has been designed to be both flexible and simple to operate and with possibilities for future extension. To achieve this, the ACS uses a simple flow language provided in XML files which indicate a series of Activities to be executed in a sequence. This allows the flows to be configured without necessarily having to write code.
The ACS implements the functional requirements for Expeditor integrator based (custom) solutions. It provides data transmission capabilities for the integration of systems which are connected to the Expeditor integrator runtime machine (e.g. PoS systems connected to the retail store server). In the retail scenario this would include for example:
Transmitting files as messages reliably from the Store Server/4690 POS Controller to the AI messaging back-end (e.g. transaction log files)
Transmitting files as messages from the AI back-end to the Store Server and/or the 4690 POS Controllers (e.g. price update files)

These functions must be put in a reliable transactional context.

Figure 2: Expeditor integrator components (Components in dotted frame are not delivered with this version)

IBM J9 DRE:
The IBM J9 is a Virtual Machine that implements the Java specification (Development Runtime Environment available through IBM Lotus Expeditor for Desktops). It is available for a variety of operating systems.
The Lotus Expeditor Client for Desktops is shipped with jclDesktop for Windows XP/Vista that is an optimized runtime environment for desktop use that enables Java developers to write applications in a "fit for purpose" runtime. There is also an optional runtime for JSE compliant program code that can be installed and licensed as a separate product. This optional Lotus Expeditor Client runtime DRE is used as base for the Expeditor integrator, because the IBM WebSphere® MQ® Client Java library and the JET require libraries that comply with the JSE standard.

OSGi® Service Platform:
OSGi is a production-ready Java environment for running and managing systems and application bundles across applications in a single JVM. The OSGi model is instantiated in IBM Lotus Expeditor Client (Please, refer to Ref_17 and Ref_18 for more details.).

Lotus Expeditor micro broker:
The IBM Lotus Expeditor micro broker acts as a message hub for clients connected to it. These clients can send messages to each other. Clients can run in the same process (Virtual Machine (VM) that implements the Java™ specifications) as the micro broker, in a different process on the same computer, or even a different computer providing a powerful messaging infrastructure. See Ref_12 for more details.

Custom Event Service (CES):
The CES is an independent component (bundle) that listens for OSGi Framework events that are fired by the Expeditor integrator bundles using OSGi Event Admin. The Custom Event Service monitors the OSGi Event Admin for certain configurable messages, converts them to Common Base Events (CBE) and forwards them to the AI messaging back-end of the Central Systems by using the Lotus Expeditor micro broker (for example, by using the Expeditor integrator’s EventQ, Business Events carrying standardized Common Base Events can be sent to the back-end).

Custom Log Service:
All Expeditor integrator bundles are using the standard OSGi Log Service for logging status information. This enables them to run on every OSGi standard platform without needing an application dependent log service. The Custom Log Service is an independent component that recognizes log entries written by the Expeditor integrator bundles by listening to Log Events fired by the OSGi Log Service. Depending on the Custom Log Service’s configuration, certain log entries are forwarded by the Custom Log Service as Log Messages to the central back-end (Monitoring System), for example by using Expeditor integrator’s LogQ. The log entry is converted in the standard CBE format so that a monitoring process at the central back-end is able to retrieve these Log Events and provide detailed status and health information about the Expeditor integrator to the Administrator or other monitoring services.

The Expeditor integrator Tivoli Monitor Service (iTMS):
The iTMS component is operating in the same way as the Custom Log Service, but does not convert the captured Log entries into CBE and forward them to the configured messaging server in the back-end. The iTSM uses the Event Integration Facility (EIF), which is provided with the Tivoli® Enterprise Console (TEC). The EIF is exploited for the creation of TEC events out of the OSGi log entries and for forwarding them to the TEC.

Custom Config Service (CCS):
The CCS extends the OSGi Configuration Admin Service interface with the capability of retrieving configuration data through either

the local Expeditor integrator configuration files (XPDinteg.xml) or through
server-managed configuration updates using the OSGi Configuration Admin service directly (e.g. IBM Lotus Expeditor server, see Ref_13).

Ref_15

The Resource Adapter Interface provides a common interface to the File System Adapter, the FTP/SSH Adapter, the JMS Destination Adapter, the Database/JDBC Adapter and other future adapters to deliver transparent resource access services. The Resource Rdapters can be registered with the Resource Monitor Service. The Resource Monitor Service uses the OSGi Event Admin service to inform interested parties (bundles) about certain events that occurred at the Resource Adapters (e.g. when a new message arrives or a file was created on the local file system).

(Local) File System Adapter:
The File System Adapter provides simple file handling capabilities independently of the used file system. The connector provides the following functions:

Retrieve a directory listing
Write a file to the local file system
Read file from local file system (regular expressions can be used to select files)
Monitor files and directories

Enhanced FTP Adapter:
The Enhanced FTP Adapter provides FTP client capabilities for transmitting files between Expeditor integrator runtime server using the FTP, sFTP or SSH protocol (e.g. between the retail store server and the 4690 POS Controllers). The FTP/SSH Adapter provides the following functions:

Retrieve a directory listing of the remote directory
Send a file to the remote file system (FTP/SSH put)
Retrieve file from local file system (FTP/SSH get: regular expression can be used to select files)
Monitor files and directories

Database (JDBC) Adapter:
The Database Adapter provides access to local and remote databases which can be accessed with JDBC drivers. The Database Adapter offers:

Read and write database records using SQL and Expeditor integrator command language (insert, update, delete records)
Take XML file content and write into database with XML support; retrieve XML content form database and send as message to back-end
Monitor database tables and records

Resource Monitor Service (RMS):
The RMS is responsible for monitoring resources local to the Expeditor integrator runtime and emitting particular events for consumption by the Expeditor integrator Application Control Service (ACS) and other interested components via the OSGi Event Admin service. Resources might represent files or directories on a file system, queues in a messaging server, database entries and so on. Operations over resources such as a read or a write are considered transactional and as such each operation provides an XAResource implementation so that the operation can be enlisted as part of a transactional unit of work. Within the Expeditor integrator system, there is a single Resource Monitor Service that monitors a number of Resource Adapters.

REST Adapter:
The REST Adapter provides the optional HTTP access channel besides the messaging protocol to the Expeditor integrator runtime. Expeditor integrator offers a specific HTTP GET request schema which must be used for accessing the REST Adapter and invoking ACS flows.
The REST Adapter converts the incoming request data into Expeditor integrator messages and sticks them into any local integrator queue. From there on, the Expeditor integrator works in the same way as for incoming messages (and processes these messages).

Queue Dispatcher (QDP):
The QDP monitors the default Expeditor integrator incoming request queue (ReqInQ) for messages and dispatches them to local queues regarding to the type of message received. This avoids blocking of new threads by other running threads which are processing long message sequences (e.g. building a file from a sequence of messages which are received over a longer period of time). It also moves messages for the back-end system from the local queues to the bridged corresponding remote back-end queues.
The dispatching algorithm, e.g. local target queues, is also configurable.

Optional: Expeditor Client Management:
The IBM Lotus Expeditor Client Management server allows for remote management of both Expeditor integrator features and plug-ins as well as resources such as configuration files. Additional tasks supported by the Expeditor Client Management server are documented in the Lotus Expeditor integrator Administrator's guide (see Ref_1).

Example Business Use Cases for Retail - Overview

For the initial Expeditor integrator release, three main Business Use Cases were especially addressed:

BUC_1: Master Data Load/Update
Master data is sent by the back-bone service bus (BSB at the back-end) to the store (e.g. price updates for PoS systems).
There are business flows in the BSB which create messages carrying data for the PoS systems of the stores in their payload. These messages are once-and-only-once delivered to the Expeditor integrator running at the remote location (e.g. on the retail store server). Then, the payload data of the messages need to be extracted from the messages, written to files in a given format, encoding etc. and must be finally transferred to their endpoint. In this use case, the 4690 FTP Server is the final destination for the transferred files. Other endpoints could be the local file system (see BUC_3), any SSH File Server or a database.
BUC_2: Transaction Data Transfer
The back-end service bus (BSB at the back-end) receives files from the remote location stores (e.g. files containing sales transactions are retrieved from a retail store).
This use case represents the previous use case in inverse order. The existence of a given file(s) or database record triggers their retrieval through the Expeditor integrator. The Expeditor integrator takes the file (database record) content and places it into messages for the back-bone service bus. These messages are then securely transferred from the remote location (e.g. retail store server) to the back-end queue where they can trigger further business processing.
BUC_3: Scale/Label Data Transfer
Data for scales and label printers is sent from the back-end service bus to the remote location (retail stores).
This use case is similar to the first one. The major difference is that the final endpoint for the files, which are created by the Expeditor integrator out of the received BSB messages, is the Store Server’s local file system.

These use cases are created by defining combinations of Expeditor integrator Application Control Service Activities (ACS Activities). These combinations are referred to as “Flows” which are executed in the Expeditor integrator Application Control Service. The flow configurations are stored in flow definition files and can easily be adapted to new business scenarios. Each flow is triggered by a given event that can be fired by another application component, e.g. a certain Resource Adapter.

Furthermore, the Expeditor integrator offers a set of ACS Activities that can be linked within flows to create new use case implementations (see chapter 4.2 Building the Flow Definitions for Business Use Cases for more details).

References

This document is based on and refers to the following documents:

Ref_1 S. Fassmann (IBM), A. Dannhauer (IBM), Installing and Configuring the IBM Lotus Expeditor integrator platform, “AdministratorsGuide_vxx.doc”, solution deployment artifact
Ref_2 S. Fassmann (IBM), A. Dannhauer (IBM), Using the IBM Lotus Expeditor integrator platform, “UsersGuide_ vxx.doc”, solution deployment artifact
Ref_3 IBM Online Support, IBM micro broker JavaDoc in IBM Lotus Expeditor Wiki, http://www.lotus.com/ldd/lewiki.nsf/dx/JavaDoc_6.2.3 (24.01.2013)
Ref_4 Eclipse Consortium, Open Messaging for Machine-to-Machine and Internet of Things (MQTT v3 Clients), http://www.eclipse.org/paho/ (14.02.2013)
Ref_5 IBM developerWorks, MQ Telemetry Transport (MQTT) V3.1 Protocol Specification, http://www.ibm.com/developerworks/webservices/library/ws-mqtt/index.html (14.02.2013)
Ref_6 IBM Redbook press, Building Smarter Planet Solutions with MQTT and IBM WebSphere MQ Telemetry, http://www.redbooks.ibm.com/redpieces/abstracts/sg248054.html?Open (14.02.2013)
Ref_7 IBM developerWorks, Extending Java Message Service messaging to retail store devices, http://www.ibm.com/developerworks/lotus/documentation/retailjms/ (14.02.2013)
Ref_8 IBM developerWorks, Messaging security guide for IBM Lotus Expeditor 6.2.3 integrator software, http://www.ibm.com/developerworks/lotus/documentation/expeditorsecurity/ (14.02.2013)
Ref_9 Oracle online, Java Message Service Specification v1.0/1.1 (JMS), http://www.oracle.com/technetwork/java/docs-136352.html (14.02.2013)
Ref_10 IBM developerWorks, IBM Patterns for e-Business, http://www-128.ibm.com/developerworks/patterns/, 16.06.2008
Ref_11 IBM Redbook press, Patterns: SOA Client - Access Integration Solutions, http://www.redbooks.ibm.com/redpieces/abstracts/sg246775.html?Open, 16.06.2008
Ref_12 IBM Online, IBM Lotus Expeditor overview, http://www-01.ibm.com/software/lotus/products/expeditor/ , 19.11.2008
Ref_13 IBM Online Support, IBM Lotus Expeditor Wiki, http://www.lotus.com/ldd/lewiki.nsf , 24.11.2011
Ref_14 IBM Online Support, IBM Lotus Expeditor integrator Wiki, http://www.lotus.com/ldd/lewiki.nsf/xpViewCategories.xsp?lookupName=Expeditor%20Integrator , 24.11.2011
Ref_15 IBM Online, IBM WebSphere Premises Server overview, http://www-01.ibm.com/software/integration/premises_server, 29.09.2008
Ref_16 IBM WebSphere Premises Server information center, http://publib.boulder.ibm.com/infocenter/pvcsensa/v6r1m0/index.jsp, 29.09.2008
Ref_17 OSGi Consortium, http://www.osgi.org , 16.06.2008
Ref_18 Eclipse Consortium, http://www.eclipse.org , 16.06.2008
Ref_19 IBM Developer Works Library, Autonomic Computing with CBEs, http://www-128.ibm.com/developerworks/autonomic/library/ac-edge11/, 16.06.2008
Ref_20 Canonical Situation Data Format: The Common Base Event V1.0.1, http://www.eclipse.org/tptp/platform/documents/resources/cbe101spec/CommonBaseEvent_SituationData_V1.0.1.pdf, 16.06.2008
Ref_21 Microsoft Help and Support Site, Installation for a User-Defined Service, http://support.microsoft.com/kb/137890, 09.11.2007
Ref_22 Microsoft Downloads for Windows 2000 and 2003 Server, http://www.microsoft.com/windows2000/remove404.mspx, 16.06.2008
Ref_23 Microsoft Download Center, Windows 2003 Resource Kit Tools, http://www.microsoft.com/downloads/details.aspx?familyid=9d467a69-57ff-4ae7-96ee-b18c4790cffd&displaylang=en, 16.06.2008
Ref_24 Apache Group, Apache Commons project, http://commons.apache.org/ , 13.08.2008
Ref_25 JCraft Inc., Java Secure Channel implementation, http://www.jcraft.com/jsch/ , 13.08.2008
Ref_26 The dojo foundation, dojo Open Source project, http://www.dojotoolkit.com/ , 16.03.2009
Ref_27 MIME type reference, http://www.w3schools.com/media/media_mimeref.asp , 27.07.2009
Ref_28 Apache HTTP client reference, http://hc.apache.org/httpclient-3.x/userguide.html , 27.07.2009

Notation

The graphical notation used in this document is adopted from Ivar Jacobson(Ivar Jacobson et al., ‘Object-oriented software engineering - A use case driven approach’, Addison-Wesley, 1992.) , supplemented by the use of a distinct symbol to distinguish non-human actors (i.e., other systems or external hardware) from human actors.